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ABSTRACT 



This review of state higher education assessment policies 
sought to test assumptions and conclusions reached during an initial phase of 
the study, to explore the dynamics of the process by which assessment becomes 
a state-level issue or concern, and to better understand the multifaceted 
relationship between state assessment policy and the improvement of teaching 
and learning. A questionnaire mailed to all 50 state academic officers in 
December 1997, with follow-ups throughout 1998, resulted in 38 responses. The 
21- item questionnaire covered: policy context, objectives, process, outcomes, 
evaluation, and future directions. The study found that while all states 
sought to satisfy a variety of objectives through their assessment policies, 
it also found significant divergence between stated policy objectives and 
reported outcomes. Campus executive officers were the most consistently 
significant players in the assessment policy process. In examining the 
relationship between policy and improvement in teaching and learning, most 
academic officers saw improved teaching and learning and assessment policy as 
highly interdependent, although there was some perception that resistance at 
the institutional and/or faculty level hindered the effectiveness of state 
policies. Future case studies will be based on the findings and responses 
from the questionnaire. Appended is the State Higher Education Assessment 
Questionnaire. (CH) 
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Methodology 



Findings from the first-year literature review, state assessment policy documents, and a 
focus group conducted with state higher education academic officers provided the 
framework for the State Higher Education Assessment Questionnaire, or SHEAQ (see 
Appendix). The SHEAQ had three primary goals: (1) to test the assumptions and con- 
clusions reached during the first year; (2) to explore the dynamics of the policy process 
by which assessment becomes a state-level issue or concern; and (3) to understand 
better the multifaceted relationship between state assessment policy on one hand and 
the improvement of teaching and learning on the other. 

The questionnaire was first mailed to all 50 state academic officers in December 1997. In 
order to achieve a higher response rate, repeated follow-ups were carried out through- 
out 1998. Although efforts continue to secure responses from a handful of missing 
states, the data in this paper reflects the responses of 38 state academic officers. 1 The 
questionnaire contained twenty-one items, divided into six sections — policy context, 
policy objectives, policy process, policy outcomes, policy evaluation, and future direc- 
tions. This section of the paper describes the thinking and reasoning that informed the 
construction of each section of the questionnaire. 

Policy Context 

In terms of the context questions, the policy documents that informed the development 
of the SHEAQ were collected during the summer and fall of 1996. Since one of the goals 
of the project is to observe and report on the evolution of state-level assessment policies, 
the context questions were designed to identify changes, if any, in a state's policy since 
1996. Documentation related to the policy change was requested. 

Another dimension of the policy context addressed in the questionnaire was gover- 
nance structure. Questions about a state's governance structure were included to allow 
us to discern what relationships, if any, exist between the nature of state assessment 
policy and governance structure. The taxonomy of state governance structures used in 
the questionnaire was drawn from McGuinness et al., who categorize state structures 
into three broad categories: (1) consolidated governing boards, (2) coordinating boards, 
and (3) planning agencies. 2 

Policy Objectives 

In the area of policy objectives, a very important lesson learned from our research dur- 
ing the first year was that states seek to accomplish a wide range of objectives with their 
assessment policies and practices. We attempted to capture this variety in our first year 
by adapting existing public policy typologies to an analysis of state assessment policy 
documents. 3 The resulting policy typology consisted of four categories, each reflecting 
the ultimate objective, or goal, of the assessment policy: regulation, reform, quality 
assurance, and accountability. 
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Our working definitions for each policy type were as follows: a regulation policy is 
designed to encourage or ensure compliance with state-mandated regulations; a reform 
ft policy is designed to encourage some type of reform (the source of this reform may 

vary); a quality assurance policy is designed to ensure that quality, as defined and 
measured by the state, is assured; and an accountability policy is designed to make 
higher education institutions more accountable to a higher authority, generally the state 
legislature. In some cases, state policies had elements of two or more of these types, while 
ft in other cases, none of the four policy types accurately or completely reflected a state's 

expressed policy objectives. 

Thus, in the second-year questionnaire, we revisited this important issue of policy type 
using a different approach. Drawing on our review and analysis of the assessment 
ft policy documents from the states, we compiled a list of all policy objectives stated in the 

• state documents. There were nine assessment policy objectives in all: (1) increasing 
accountability to the public, (2) increasing fiscal accountability, (3) improving teaching, 
(4) improving student learning, (5) promoting planning on campus, (6) improving 
academic program efficiency, (7) facilitating intrastate comparisons, (8) facilitating 
ft interstate comparisons, and (9) reducing academic program duplication. 

If a state's policy was designed to meet more than one objective, it was not necessarily 
clear if all of the multiple objectives were of equal weight, or if certain objectives were, 
in fact, more significant than others. Therefore, we also asked the state academic officers 
ft to evaluate the relative significance of each objective. Finally, given the project's focus 

on the issues of improvement of teaching and learning, we provided the academic 
officers with an open-ended format for discussing the relationship between these issues 
and state assessment policy. 

ft Policy Process 

The policy process section of the questionnaire represented the first attempt to apply a 
public policy process framework to a discussion of higher education assessment policies 
at the state level. The process framework was adapted from Anderson et al., who de- 
ft scribe the policy process as "the various activities by which public policy is formed." 4 

Anderson and his colleagues articulate a six-stage conceptual framework as a means of 
analyzing the policy process in general, while acknowledging the differences in pro- 
cesses against content areas. "Welfare policy, natural resource regulation, economic 
stability policy, and civil rights policy are distinguished by different processes.... This 
ft does not mean that the environment of each policy area is unique in such a way that 

precludes generalizations about public policy." 5 



ft 




National Center for Postsecondary Improvement 



Page 3 



While education policy in general, and higher education policy in particular, are also 
distinguished by different processes as a policy domain, and these processes in turn 
differ by state, the stages described by Anderson and his colleagues are useful tools in 
researching state-level assessment policy. The stages are detailed here. 

1. problem formation: "relief is sought from a situation that produced a human 
need, a deprivation, or dissatisfaction." (For the purposes of our study, the 
first two stages in the Anderson model — problem formation and policy 
agenda — were combined into one stage representing the period when the 
need for a state-level assessment policy was first recognized.) 

2. policy formulation: "pertinent and acceptable proposed courses of action for 
dealing with public problems are developed." 

3. policy adoption: "support is developed for a specific proposal such that the 
policy is legitimized or authorized." 

4. policy implementation: "the application of the policy to the problem." 

5. policy evaluation: "an attempt is made to determine whether or not the 
policy has been effective." 

In adapting this framework to the analysis of state-level assessment policies, we were 
also interested in identifying the most significant entities at each of these five stages. So 
we added another dimension to this framework: a list of twelve entities or influences, 
compiled from the literature review and policy analysis in the first year, that play some 
role in the assessment policy process at the state level. These twelve entities are: (1) state 
legislatures, (2) governors /executive staffs, (3) executive agencies, (4) system boards, (5) 
campus executive officers, (6) faculty, (7) external consultants, (8) existing policies and 
practices on campuses, (9) other states' assessment policies and practices, (10) professional 
organizations such as SHEEO and AAHE, (11) regional accrediting associations, and (12) 
disciplinary accrediting associations. Academic officers were asked to evaluate the relative 
significance of each of these entities at each of the five stages in the policy process model. 

Policy Outcomes 

Based on the first year's research, we realized that while a state may have certain objec- 
tives for its assessment policy, those objectives may not always be met. Alternatively, the 
policy may have unintended or unexpected outcomes. This distinction between policy 
objectives and outcomes is a critical one, as we attempt to understand the dynamics of 
the policy process and make practical recommendations to states on how to achieve the 
objectives they set for their assessment policies. 
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The distinction between objectives and outcomes is also addressed in the policy analysis 
literature. An effort has been made to distinguish between intentional analysis, which 
focuses on what was, or is, intended by a policy, and functional analysis, which focuses 
on what actually happened as a result of the policy. 6 For the purposes of this question- 
naire and its results, we regard objectives as the key to our intentional analysis, and 
outcomes as the key to our functional analysis. 

Policy Evaluation 

This section was designed to determine whether states were successful in improving 
teaching and learning through the use of assessment policies, and if so, what reasons the 
academic officers offered for that success. Conversely, if states were unsuccessful, we 
wanted to understand the reasons for failure. This section also included an item asking 
academic officers to comment on the possible reasons for a divergence between policy 
objectives and outcomes, provided such a divergence existed. 

Given that the primary emphasis of our research is the use of assessment to improve 
teaching and learning, this section of the questionnaire also asked a series of open- 
ended questions designed to give state academic officers an opportunity to reflect and 
comment upon the relationship between their states' assessment policies and the im- 
provement of teaching and learning. 

Future Directions 

This section, which asked academic officers to list any institutions in their state that 
were doing interesting or innovative assessment work, was designed primarily to assist 
Project 5.2 and 5.3 in determining possible candidates for their institutional case studies. 
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Results 



According to the state academic officers, the most common objective of state-level 
assessment policies was increasing public accountability (see Table 1). A very close 
second was improving learning. Improving teaching was the third most common as- 
sessment policy objective. The least common policy objectives were facilitating intrast- 
ate comparisons and reducing academic program duplication. 

Table 1. State Assessment Policy Objectives 



Policy Objective 


Number of States (n=38) 


increasing public accountability 


33 


improving learning 


30 


improving teaching 


28 


promoting planning on campus 


24 


improving academic program efficiency 


18 


increasing fiscal accountability 


15 


facilitating interstate comparisons 


8 


facilitating intrastate comparisons 


7 


reducing academic program duplication 


7 


other 


4 



The relative significance of these policy objectives matched closely with the reported 
frequencies. Increasing public accountability had the highest average significance score, 
followed in order by improving student learning, improving teaching, and promoting 
planning on campuses. Interestingly, the range of significance scores was small; the 
highest average significance score was roughly 3.7 (on a scale of one to four), and the 
lowest average score was 2.5, which suggests that there are no particular policy objec- 
tives that enjoy substantially more weight than others. 

The most common outcome of states' assessment policies was, not surprisingly, increas- 
ing public accountability (see Table 2). More unexpected, however, was that promoting 
planning on campus ranked second, followed by improving teaching, improving stu- 
dent learning, and improving academic program efficiency. 
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Table 2. State Assessment Policy Outcomes 



Policy Outcome 


Number of states (n=38) 


increasing public accountability 


31 


promoting planning on campus 


27 


improving teaching 


22 


improving learning 


20 


improving academic program efficiency 


15 


reducing academic program efficiency 


10 


facilitating interstate comparisons 


6 


facilitating intrastate comparisons 


6 



During the problem formation stage of the policy process, state academic officers 
ranked the system boards as the most significant, followed very closely by state legisla- 
tures and campus executive officers (see Table 3). Least significant at this stage are 
disciplinary accrediting associations and external consultants. 

Table 3. Significance of Entities in Problem Formation Stage 



Entity 


Significance (1-4 point scale) 


system boards 


2.8 


state legislature 


2.7 


campus executive officers 


2.5 


governor/executive staff 


2.4 


faculty 


2.3 


executive agencies 


2.3 


existing campus practices 


2.2 


regional accrediting association 


2.2 


professional organizations 


1.8 


other states' policies 


1.8 


external consultants 


1.6 


disciplinary accrediting associations 


1.5 



State academic officers regard campus executive officers as most significant during the 
policy formulation stage, when a variety of policy options are put forth and debated at 
the state level (see Table 4). Nearly as significant were system boards. The state legisla- 
ture and faculty were tied at this stage for third. 
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Table 4. Significance of Entities in Policy Formulation 



Entity 


Significance 


campus executive officers 


3.2 


system boards 


3.1 


state legislature 


2.5 


faculty 


2.5 


existing campus practices 


2.4 


executive agencies 


2.3 


other states' policies 


2.2 


govemor/executive staff 


2.0 


regional accrediting associations 


2.0 


professional organizations 


1.7 


external consultants 


1.7 


disciplinary accrediting associations 


1.5 



Campus executive officers are also ranked as the most significant players during the policy 
adoption stage, according to the state academic officers (see Table 5). Following campus 
chiefs in terms of significance at this stage are system boards, faculty, state legislatures, and 
existing campus practices. There is a fairly sharp drop-off, however, between the system 
boards and the faculty. 

Table 5. Significance of Entities in Policy Adoption 



Entity 


Significance 


campus executive officers 


3.4 


system boards 


3.3 


faculty 


2.5 


state legislature 


2.4 


existing campus practices 


2.1 


governor/ executive staff 


2.0 


regional accrediting associations 


2.0 


executive agencies 


2.0 


other states' policies 


1.8 


professional organizations 


1.6 


external consultants 


1.4 


disciplinary accrediting associations 


1.4 



During the policy implementation stage, when the "rubber hits the road" and an assess- 
ment policy is put into effect, campus executive officers are again the most significant 
(see Table 6). Faculty, as one might expect (and hope), are ranked second, followed by 
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system boards. The substantial gap between campus executive officers and faculty is 
worth noting here. 

Table 6. Significance of Entities in Policy Implementation 



Entity 


Significance 


campus executive officers 


3.8 


faculty 


3.2 


system boards 


2.8 


existing campus practices 


2.5 


executive agencies 


2.2 


regional accrediting associations 


2.1 


state legislature 


1.7 


other states' policies 


1.6 


govemor/executive staff 


1.6 


professional organizations 


1.5 


disciplinary accrediting associations 


1.5 


external consultants 


1.3 



Finally, when the time comes to determine exactly what the assessment policy is doing 
and how well it is doing it, state academic officers ascribe the most significance to campus 
executive officers once more, followed by system boards and faculty (see Table 7). The 
remainder of the entities all received an average significance score of 2.0 or below. 

Table 7. Significance of Entities in Policy Evaluation Interpretation 



Entity 


Significance 


campus executive officers 


3.2 


system boards 


3.0 


faculty 


2.6 


state legislature 


2.0 


executive agencies 


2.0 


regional accrediting associations 


1.9 


existing campus practices 


1.6 


govemor/executive staff 


1.6 


external consultants 


1.5 


disciplinary accrediting associations 


1.5 


other states' practices 


1.4 


professional organizations 


1.3 
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Interpretation 



• Based on the data presented above, it is clear that state academic officers perceive that 

their states' assessment policies seek to meet a variety of objectives. There were no 
states that reported a single assessment policy objective; all states reported at least two. 
Given the increasing demands on higher education to be responsive to a wide range of 
public and political constituencies, it is not surprising that the most frequent and most 
ft significant policy objective across the states is increasing public accountability. 

Less than half of the academic officers, however, considered increasing fiscal account- 
ability to be an objective, which demonstrates the multifaceted nature of accountability. 
Because accountability has played, and continues to play, such a prominent role in the 
ft assessment movement, understanding the components of accountability is a critical 

next step for the case study research. 

Given that all states have multiple objectives for their assessment policies, understand- 
ing the nature of the interaction between these objectives is also essential. It is likely that 
ft some of these objectives complement or facilitate each other; for example, promoting 

planning on campuses might very well lead to improving academic program efficiency, 
which could result in reducing academic program duplication. On the other hand, some 
of the objectives might work at cross-purposes to one another. If one of the policy objec- 
tives is facilitating intrastate or interstate comparisons, there exists the possibility that 
ft such comparisons will obscure the differences across academic programs, student 

populations, and institutional types, which could result in the assessment policy having 
a negative effect on teaching and learning. Studying the interactions between policy 
objectives will be another goal of the case studies. 

ft We also discovered divergence between stated policy objectives and reported policy 

outcomes. For example, while thirty state academic officers claimed that improving 
learning was a policy objective, it is an outcome in only twenty states. On the other 
hand, twenty-four states indicated that promoting planning on campus was an objec- 
tive, while twenty-seven assert it is an outcome. Clearly, there is a rich vein of research 
ft to be mined here — particularly as we attempt to understand how states can craft assess- 

ment policies that maximize the improvement of teaching and learning. 

The application of a policy process framework to the study of state-level assessment 
policy in Project 5.1 is a unique and original contribution to the scholarly literature in 
§ this field. This approach allows the policy process to be broken down into more focused 

and well-defined components, which in turn allows for a much more fine-grained 
picture of exactly how assessment becomes an issue at the state level. The analytical 
power of this process framework was supplemented by our efforts to specify the rela- 
tive significance of key entities at each stage in the process. 

ft 
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In broad terms, state academic officers report that campus executive officers are the 
most consistently significant players in the assessment policy process. In four of five 
stages, they ranked first in average significance. The relative significance of system 
boards also appears to be fairly consistent across all five stages. 

The rise and fall of other entities at different stages also suggests important patterns of 
influence. Faculty, for example, do not have as high a profile as one might expect during 
the policy formulation and adoption stages, but do emerge as key players when it 
comes to policy implementation. This might reflect the fact that faculty do not have as 
much input during the early stages of the process as they should, which could have 
implications for faculty support of state-mandated assessment activities. This also 
highlights the need for campus executive officers and system boards to serve, as they do 
with regard to many other issues, as liaisons or mediators between the state, its inter- 
ests, and its policy on the one hand, and the faculty, their interests, and their actions on 
the other. These average significance scores represent an exciting first step toward 
"unpacking" this question of relative influence of various entities in the policy process; 
more detailed exploration will come in the case studies. 

The responses to the open-ended questions afforded a less systematic, but still no less 
revealing, perspective on state-level assessment, especially the role of assessment in the 
improvement of teaching and learning. In terms of the relationship between state policy 
and improvement, many state academic officers regarded the link as implicit. Often, the 
improvement of teaching was the result of the assessment policies designed to achieve 
other, more explicit, goals. But does the implicit nature of this relationship mean that it 
does not receive the attention it deserves from policymakers? 

Other academic officers pointed out that feedback from assessment can lead to an im- 
provement in teaching, since it gives faculty and institutions a chance to find out which 
teaching strategies are successful and which are not. The effect of this feedback is en- 
hanced by networking, whether formal or informal, among faculty who meet at state 
conferences. Other factors described by a few academic officers as enhancing the im- 
provement of teaching include ties to regional accrediting associations and institutional 
autonomy from excessive state regulation and standardization. 

Most academic officers saw improved teaching and learning as highly interdependent, 
certainly as policy objectives. A lack of money, however, jeopardizes both goals. Low 
faculty salaries in some less-affluent states, decreasing state appropriations for public 
higher education generally, and the dearth of funding for assessment activities in par- 
ticular were symptomatic of the financial predicament. 

Another broad theme that emerged across the responses was the perception of state 
academic officers that resistance at the institutional and/or faculty levels hindered the 
effectiveness of states' assessment policies. Faculty who are unwilling to consider how 
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students learn in new and perhaps unconventional ways was cited as an obstacle, as 
was the "limited sophistication of personnel," primarily at smaller institutions. Of 
course, this is a rather provocative indictment of the faculty that merits further consid- 
eration and a chance for the faculty to respond to the criticism. 

It is also widely held by academic officers, as well as by many scholars in the assess- 
ment field, that many assessment policies are problematic because of the difficulties 
associated with developing, much less perfecting, meaningful and valid measures of 
teaching and learning. The absence of a common understanding of adequate mea- 
sures — across disciplines and across institutions — remains a barrier to the improvement 
of teaching and learning. 



Conclusion 

Just as our research from the first year informed the construction of our questionnaire in 
the second year, our analysis of the responses from the questionnaire will inform the 
development of our case studies. Having touched on some of the key issues regarding 
state-level assessment policies and the relationship between these policies and the 
improvement of teaching and learning with the questionnaire, we will be able to ex- 
plore these issues in much greater depth when we visit states and conduct our case 
study research. We will also draw substantially upon the literature review that makes 
up the first part of this report, in order to understand the role of the state in the gover- 
nance of higher education, the relationship between state regulation and institutional 
autonomy, and the effects of differing governance structures on higher education policy. 
All of this contributes to our ultimate goal: the development of practical assessment 
policy models for states to consider as they try to craft effective higher education assess- 
ment policy and legislation. 
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Appendix: State Higher Education Assessment Questionnaire (SHEAQ) 













State Higher Education Assessment Questionnaire (SHEAQ) 

National Center for Postsecondary Improvement 

University of Michigan * Room 2339 School of Education * Ann Arbor, MI 48109-1259 
John J.K. Cole Michael T. Nettles Sally Sharp 
313-647-1654 (phone) 

313-936-2741 (Fax) 

This survey consists of twenty-one questions about the practice and impact of state assessment policies. 
While the primary emphasis of our work is the impact of student outcomes assessment on the improvement 
of teaching and learning, we want to place this impact in a broader, comparative context. The survey is 
divided into six sections: (1) context, (2) objectives, (3) processes, (4) outcomes, (5) evaluation, and (6) future 
directions. There are both open- and close-ended questions, with space provided for answers after each 
open-ended question. If you find that the space provided for your answer is not sufficient, please feel free 
to write on the back of the page, indicating the number of the question you are answering. If you have 
questions about this survey, please call Michael Nettles, John Cole or Sally Sharp at the National Center for 
Postsecondary Improvement at the University of Michigan, at 313-647-1654. 

This survey is being administered to the state higher education academic officers in each of the 50 states. 
Your responses will be combined with those of other states and where possible will be reported as group 
averages. Because some of your responses may identify actions, programs, and relationships unique to 
your state, we are unable to assure you of the complete confidentiality of your responses. We are very 
grateful for your time and your willingness to complete this survey; we hope the answers from all 50 states, 
when compared and analyzed, will help both institutions and policymakers as they address issues of 
assessment. We will provide you with a draft of our report to give you an opportunity to react to our 
analysis. 

In this space, please identify the person and his/her title who completed this questionnaire. If more than 
one person collaborated on this questionnaire, please identify each person and his/her title. 

Name: Title: 



Context 

1. Have there been any changes in [INDIVIDUAL STATE NAME] assessment policy since August 1996? 

Yes No 

2. Have there been any new policies relating to assessment implemented in your state since August 1996? 

Yes No 

If your answer is “yes” to either question, please attach documentation describing the changes and/or the new 
policy, and base your answers to the following questions on the most recent policy or policies. 

3. How would you characterize the governance structure for public higher education in [INDIVIDUAL STATE 
NAME]? Please mark one only . (Classification system taken from McGuinness et al., 1994) 

consolidated governing board for all institutions 

consolidated governing board for all senior institutions and a separate board for junior/ 

community colleges 

regulatory board with program approval authority 

advisory board with program review and recommendation authority 

planning agency 

other: 
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Objectives 



4. What is the current objective of [INDIVIDUAL STATE NAME] assessment policy? (Please mark all that 
apply.) 



increasing accountability to public 
increasing fiscal responsibility 
improving teaching 
improving student learning 
promoting planning on campuses 
improving academic program efficiency 
facilitating intra state comparisons 
facilitating inter state comparisons 
reducing academic program duplication 
other: 



5. Given what you have marked as the objective(s) of [INDIVIDUAL STATE NAME] assessment policy, please 
rate the significance, as you see it, of each of these objectives on a four-point scale, with “1” representing “not 
significant” and a “4” representing “very significant .” Please circle one number for each item checked in question 
#4. 



Not 

objective 

increasing accountability to the public 1 

increasing fiscal accountability 1 

improving teaching 1 

improving student learning 1 

promoting planning on campuses 1 

improving academic program efficiency 1 

facilitating intrastate comparisons 1 

facilitating interstate comparisons 1 

reducing academic program duplication 1 

other: 1 



Significance 

Slightly Moderately Very 



2 

2 

2 

2 

2 

2 

2 

2 

2 

2 



3 

3 

3 

3 

3 

3 

3 

3 

3 

3 



4 

4 

4 

4 

4 

4 

4 

4 

4 

4 



6. Since one of our areas of emphasis is the impact of assessment on the improvement of teaching and learning, how 
would you characterize the relationship between [INDIVIDUAL STATE NAME] assessment policy and the 
improvement of teaching ? What, if anything, about your policy demonstrates a commitment to the improvement of 
teaching ? 
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7. Since one of our areas of emphasis is the impact of assessment on the improvement of teaching and learning, how 
would you characterize the relationship between [INDIVIDUAL STATE NAME] assessment policy and the 
improvement of learning ? What, if anything, about your policy demonstrates a commitment to the improvement of 
learning ? 



In this section, we are seeking to understand the stages of the policymaking process, and what entities play the most 
important roles during each stage. We are considering five stages in our analysis, so there are five questions, one for 
each stage. For greater clarity, a definition is provided for each stage as part of the questions. (Stages and 
definitions adapted from Anderson et al., 1984) 

8. On a four-point scale, with a “1” representing “not significant” and a “4” representing “very significant ,” please 
rate each of the following entities in terms of their relative significance in the problem formation stage of 
[INDIVIDUAL STATE NAME] assessment policy. During the problem formation stage, “relief is sought from a 
situation that produces a human need, a deprivation, or dissatisfaction.” Please circle one number for each entity. 



Processes 



Significance 



Not 



Slightly 



Moderately Very 



entities 

state legislature 

govemor/executive staff 

exec, agencies (e.g., state ed. department) 

system boards of trustees/regents 

campus executive officers 

faculty 

external consultants 

existing policies and practices on campuses 
other states’ policies and practices 
professional orgs. (e.g., SHEEO; AAHE) 
regional accreditation association 
disciplinary accreditation associations 

other: 

other: 



2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 



3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 



4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 
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9. On a four-point scale, with a “1” representing “not significant” and a “4” representing “very significant .” now 
please rate each of the following entities in terms of their relative significance in the policy formulation stage of 
[INDIVIDUAL STATE NAME] assessment policy. During the policy formulation stage, “pertinent and acceptable 
proposed courses of action for dealing with public problems” are developed. Please circle one number for each 
entity. 



Not 

entities 

state legislature 1 

govemor/executive staff 1 

exec, agencies (e.g., state ed. department) 1 

system boards of trustees/regents 1 

campus executive officers 1 

faculty 1 

external consultants 1 

existing policies and practices on campuses 1 

other states’ policies and practices 1 

professional orgs. (e.g., SHEEO; AAHE) 1 

regional accreditation association 1 

disciplinary accreditation associations 1 

other: 1 

other: 1 



Significance 

Slightly Moderately Very 



2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 



3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 



4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 



10. On a four-point scale, with a “1” representing “not significant” and a “4” representing u verv significant .” please 
rate each of the following entities in terms of their relative significance in the policy adoption stage. During the 
policy adoption stage, ‘‘support is developed for a specific proposal such that the policy is legitimized or 
authorized.” Please circle one number for each entity. 





Not 


Slightly 


Significance 

Moderately 


Verv 


entities 

state legislature 


1 


2 


3 


4 


govemor/executive staff 


1 


2 


3 


4 


exec, agencies (e.g., state ed. department) 


1 


2 


3 


4 


system boards of trustees/regents 


1 


2 


3 


4 


campus executive officers 


1 


2 


3 


4 


faculty 


1 


2 


3 


4 


external consultants 


1 


2 


3 


4 


existing policies and practices on campuses 


1 


2 


3 


4 


other states’ policies and practices 


1 


2 


3 


4 


professional orgs. (e.g., SHEEO; AAHE) 


1 


2 


3 


4 


regional accreditation association 


1 


2 


3 


4 


disciplinary accreditation associations 


1 


2 


3 


4 


other: 


1 


2 


3 


4 


other: 


1 


2 


3 


4 
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1 1 . On a four-point scale, with a “1” representing “not significant” and a “4” representing “very significant .” please 
rate each of the following entities in terms of their relative significance in the policy implementation stage. During 
the policy implementation stage, there is the “application of the policy to the problem ” Please circle one number 
for each entity. 



Not 



entities 

state legislature 1 

govemor/executive staff 1 

exec, agencies (e.g., state ed. department) 1 

system boards of trustees/regents 1 

campus executive officers 1 

faculty 1 

external consultants 1 

existing policies and practices on campuses 1 

other states’ policies and practices 1 

professional orgs. (e.g., SHEEO; AAHE) 1 

regional accreditation association 1 

disciplinary accreditation associations 1 

other: 1 

other: 1 



Significance 

Slightly Moderately Very 



2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 



3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 



4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 



12. On a four-point scale, with a “1” representing “not significant” and a “4” representing “very significant ,” please 
rate each of the following entities in terms of their relative significance in the policy evaluation stage. During the 
policy implementation stage, “an attempt is made to determine whether or not the policy has been effective.” Please 
circle one number for each entity. 



Not 



entities 

state legislature 1 

govemor/executive staff 1 

exec, agencies (e.g., state ed. department) 1 

system boards of trustees/regents 1 

campus executive officers 1 

faculty 1 

external consultants 1 

existing policies and practices on campuses 1 

other states’ policies and practices 1 

professional orgs. (e.g., SHEEO; AAHE) 1 

regional accreditation association 1 

disciplinary accreditation associations 1 

other: 1 

other: 1 



Significance 

Slightly Moderately Very 



2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 



3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 



4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 
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Outcomes 



13. There is often a distinction made between the objective(s) of a policy and the outcome(s) of a policy. Questions 
#4 and 5 explored objectives. In this question, please tell us what you think have been the outcomes of 
[INDIVIDUAL STATE NAME] policy. (Please mark all that apply.) 

increasing accountability to public 

increasing fiscal responsibility 

improving teaching 

improving student learning 

promoting planning on campuses 

improving academic program efficiency 

facilitating intra state comparisons 

facilitating inter state comparisons 

reducing academic program duplication 

other: 

other: 



14. Please circle one answer to each of the following questions. Is [INDIVIDUAL STATE NAME] currently using: 



common instruments across institutions for assessment of teaching? 
common performance indicators for assessment of teaching? 

common performance indicators for assessment of teaching that are linked to funding? 
common procedures for collecting assessment data on teaching? 

common instruments across institutions for assessment of learning? 
common performance indicators for assessment of learning? 

common performance indicators for assessment of learning that are linked to funding? 
common procedures for collecting assessment data on learning? 



Yes 


No 


Yes 


No 


Yes 


No 


Yes 


No 


Yes 


No 


Yes 


No 


Yes 


No 


Yes 


No 



15. If any of the items in question #14 is circled, please describe the impetus for the interest in/adoption for each of 
the measures circled. 
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Evaluation 



16. If there are differences between policy objectives and policy outcomes in [INDIVIDUAL STATE NAME], why 
do you think these differences exist? 



17. Provided that the improvement of teaching is an objective of [INDIVIDUAL STATE NAME] assessment 
policy, what has enhanced [INDIVIDUAL STATE NAME] success in achieving this objective? Please interpret 
this question broadly; we are trying to understand what persons, events, and/or processes enhance [INDIVIDUAL 
STATE NAME] success in achieving this objective. 



1 7a. What evidence exists of your state’s success at meeting the objective of improving teaching? 
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18. Provided that the improvement of learning is an objective of [INDIVIDUAL STATE NAME] assessment 
policy, what has enhanced [INDIVIDUAL STATE NAME] success in achieving this objective? Please interpret 
this question broadly; we are trying to understand what persons, events, and/or processes enhance [INDIVIDUAL 
STATE NAME] success in achieving this objective. 



18a. What evidence exists of your state’s success at meeting the objective of improving learning? 



19. Provided that the improvement of teaching is an objective of [INDIVIDUAL STATE NAME] assessment 
policy, what has hindered [INDIVIDUAL STATE NAME] success in achieving this objective? Please interpret this 
question broadly; we are trying to understand what persons, events, and/or processes hinder [INDIVIDUAL STATE 
NAME] success in achieving this objective. 



0 
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20. Provided that the improvement of learning is an objective of [INDIVIDUAL STATE NAME] assessment 
policy, what has hindered [INDIVIDUAL STATE NAME] success in achieving this objective? Please interpret this 
question broadly; we are trying to understand what persons, events, and/or processes hinder [INDIVIDUAL STATE 
NAME] success in achieving this objective. 



Future Directions 

21. As part of our research in Year Three, we are planning to conduct case studies on the most interesting and/or 
innovative assessment policies and practices. In your opinion, what campuses/institutions in [INDIVIDUAL 
STATE NAME] are doing the most interesting and/or innovative work related to assessment? 



Please return this survey before January 15, 1998 in the envelope provided to: 

NCPI * University of Michigan * Room 2239 School of Education * Ann Arbor, MI 48109-1259 
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